Variable resistor apparatus formed utilizing nanotechnology

ABSTRACT

A variable resistor apparatus includes a plurality of nanoparticles disposed between two terminals, wherein the plurality of nanoparticles provides an electrical resistance. An electric field applied to the plurality of nanoparticles across the two terminals results in an alignment of the nanoparticles over time and a decrease in the electrical resistance thereby providing a variable resistor apparatus. The electric or electrical field can be applied across the two terminals perpendicular to the plurality of nanoconnections. The nanoparticles can comprise nanoconductors, which can be formed as, for example, nanotubes and/or nanowires. The nanoparticles are generally disposed in a solution within a connection gap formed between the two terminals. The solution can comprise a solvent and/or a suspension of nanoparticles forming a mixture thereof. The solution can also be provided as a liquid, a gel, and or a gas. The solution may also comprise a dielectric.

CROSS REFERENCE TO RELATED PATENT APPLICATION

This patent application is a continuation of U.S. patent applicationSer. No. 10/095,273 entitled “Physical Neural Network DesignIncorporating Nanotechnology,” which was filed on Mar. 12, 2002, thedisclosure of which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

Embodiments generally relate to nanotechnology. Embodiments also relateto variable resistor components. Embodiments additionally relate toneural networks and neural computing systems.

BACKGROUND

Neural networks are computational systems that permit computers toessentially function in a manner analogous to that of the human brain.Neural networks do not utilize the traditional digital model ofmanipulating 0's and 1's. Instead, neural networks create connectionsbetween processing elements, which are equivalent to neurons of a humanbrain. Neural networks are thus based on various electronic circuitsthat are modeled on human nerve cells (i.e., neurons). Generally, aneural network is an information-processing network, which is inspiredby the manner in which a human brain performs a particular task orfunction of interest. Computational or artificial neural networks arethus inspired by biological neural systems. The elementary buildingblock of biological neural systems is of course the neuron, themodifiable connections between the neurons, and the topology of thenetwork.

Biologically inspired artificial neural networks have opened up newpossibilities to apply computation to areas that were previously thoughtto be the exclusive domain of human intelligence. Neural networks learnand remember in ways that resemble human processes. Areas that show thegreatest promise for neural networks, such as pattern classificationtasks such as speech and image recognition, are areas where conventionalcomputers and data-processing systems have had the greatest difficulty.

In general, artificial neural networks are systems composed of manynonlinear computational elements operating in parallel and arranged inpatterns reminiscent of biological neural nets. The computationalelements, or nodes, are connected via variable weights that aretypically adapted during use to improve performance. Thus, in solving aproblem, neural net models can explore many competing hypothesissimultaneously using massively parallel nets composed of manycomputational elements connected by links with variable weights. Incontrast, with conventional von Neumann computers, an algorithm mustfirst be developed manually, and a program of instructions written andexecuted sequentially. In some applications, this has proved extremelydifficult. This makes conventional computers unsuitable for manyreal-time problems.

In a neural network, “neuron-like” nodes can output a signal based onthe sum of their inputs, the output being the result of an activationfunction. In a neural network, there exists a plurality of connections,which are electrically coupled among a plurality of neurons. Theconnections serve as communication bridges among of a plurality ofneurons coupled thereto. A network of such neuron-like nodes has theability to process information in a variety of useful ways. By adjustingthe connection values between neurons in a network, one can matchcertain inputs with desired outputs.

One does not program a neural network. Instead, one “teaches” a neuralnetwork by examples. Of course, there are many variations. For instance,some networks do not require examples and extract information directlyfrom the input data. The two variations are thus called supervised andunsupervised learning. Neural networks are currently used inapplications such as noise filtering, face and voice recognition andpattern recognition. Neural networks can thus be utilized as an advancedmathematical technique for processing information.

Neural networks that have been developed to date are largelysoftware-based. A true neural network (e.g., the human brain) ismassively parallel (and therefore very fast computationally) and veryadaptable. For example, half of a human brain can suffer a lesion earlyin its development and not seriously affect its performance. Softwaresimulations are slow because during the learning phase a standardcomputer must serially calculate connection strengths. When the networksget larger (and therefore more powerful and useful), the computationaltime becomes enormous. For example, networks with 10,000 connections caneasily overwhelm a computer. In comparison, the human brain has about100 billion neurons, each of which can be connected to about 5,000 otherneurons. On the other hand, if a network is trained to perform aspecific task, perhaps taking many days or months to train, the finaluseful result can be etched onto a piece of silicon and alsomass-produced.

A number of software simulations of neural networks have been developed.Because software simulations are performed on conventional sequentialcomputers, however, they do not take advantage of the inherentparallelism of neural network architectures. Consequently, they arerelatively slow. One frequently used measurement of the speed of aneural network processor is the number of interconnections it canperform per second. For example, the fastest software simulationsavailable can perform up to about 18 million interconnects per second.Such speeds, however, currently require expensive super computers toachieve. Even so, 18 million interconnects per second is still too slowto perform many classes of pattern classification tasks in real time.These include radar target classifications, sonar target classification,automatic speaker identification, automatic speech recognition andelectro-cardiogram analysis, etc.

The implementation of neural network systems has lagged somewhat behindtheir theoretical potential due to the difficulties in building neuralnetwork hardware. This is primarily because of the large numbers ofneurons and weighted connections required. The emulation of even of thesimplest biological nervous systems would require neurons andconnections numbering in the millions. Due to the difficulties inbuilding such highly interconnected processors, the currently availableneural network hardware systems have not approached this level ofcomplexity. Another disadvantage of hardware systems is that theytypically are often custom designed and built to implement oneparticular neural network architecture and are not easily, if at all,reconfigurable to implement different architectures. A true physicalneural network (i.e., artificial neural network) chip, for example, hasnot yet been designed and successfully implemented.

The problem with pure hardware implementation of a neural network withtechnology as it exists today, is the inability to physically form agreat number of connections and neurons. On-chip learning can exist, butthe size of the network would be limited by digital processing methodsand associated electronic circuitry. One of the difficulties in creatingtrue physical neural networks lies in the highly complex manner in whicha physical neural network must be designed and built. It is believedthat solutions to creating a true physical and artificial neural networklie in the use of nanotechnology and the implementation of analogvariable connections.

The term “Nanotechnology” generally refers to nanometer-scalemanufacturing processes, materials and devices, as associated with, forexample, nanometer-scale lithography and nanometer-scale informationstorage. Nanometer-scale components find utility in a wide variety offields, particularly in the fabrication of microelectrical andmicroelectromechanical systems (commonly referred to as “MEMS”).Microelectrical nano-sized components include transistors, resistors,capacitors and other nano-integrated circuit components. MEMS devicesinclude, for example, micro-sensors, micro-actuators, micro-instruments,micro-optics, and the like.

Based on the foregoing, it is believed that a physical neural networkwhich incorporates nanotechnology is a solution to the problemsencountered by prior art neural network solutions. Additionally, it isbelieved that a variable resistor apparatus can be constructed based onnanotechnology and utilized either as an individual component forvariable resistance purposes, or in association with physical neuralnetworks, including artificial neurons and components thereof asdescribed herein.

BRIEF SUMMARY

The following summary is provided to facilitate an understanding of someof the innovative features unique to the embodiments, and is notintended to be a full description. A full appreciation of the variousaspects of the embodiments can be gained by taking the entirespecification, claims, drawings, and abstract as a whole.

It is, therefore, one aspect of the present invention to provide for avariable resistor apparatus that can be formed based on nanotechnology.

It is another aspect of the present invention to provide a physicalneural network, which can be formed from a plurality of interconnectednanoconnections or nanoconnectors.

It is a further aspect of the present invention to provide neuron likenodes, which can be formed and implemented utilizing nanotechnology;

-   -   It is also an aspect of the present invention to provide a        physical neural network that can be formed from one or more        neuron-like nodes.

It is yet a further aspect of the present invention to provide aphysical neural network, which can be formed from a plurality ofnanoconductors, such as, for example, nanowires and/or nanotubes.

The above and other aspects can be achieved as is now described. Aphysical neural network based on nanotechnology is disclosed herein,including methods thereof. Such a physical neural network generallyincludes one or more neuron-like nodes, connected to a plurality ofinterconnected nanoconnections. Each neuron-like node sums one or moreinput signals and generates one or more output signals based on athreshold associated with the input signal. The physical neural networkalso includes a connection network formed from the interconnectednanoconnections, such that the interconnected nanoconnections usedthereof by one or more of the neuron-like nodes can be strengthened orweakened according to an application of an electric field. Alignment hasalso been observed with a magnetic field, but electric fields aregenerally more practical. Note that the connection network is generallyassociated with one or more of the neuron-like nodes.

The output signal is generally based on a threshold below which theoutput signal is not generated and above which the output signal isgenerated. The transition from zero output to high output need notnecessarily be abrupt or non linear. The connection network comprises anumber of layers of nanoconnections, wherein the number of layers isgenerally equal to a number of desired outputs from the connectionnetwork. The nanoconnections are formed without influence fromdisturbances resulting from other nanoconnections thereof. Suchnanoconnections may be formed from an electrically conducting material.The electrically conducting material can be selected such that a dipoleis induced in the electrically conducting material in the presence of anelectric field. Such a nanoconnection may comprise a nanoconductor.

The connection network itself may comprise a connection networkstructure having a connection gap formed therein, and a solution locatedwithin the connection gap, such that the solution comprises a solvent orsuspension and one or more nanoconductors. Preferably, a plurality ofnanoconductors is present in the solution (i.e., mixture). Note thatsuch a solution may comprise a liquid and/or gas. An electric field canthen be applied across the connection gap to permit the alignment of oneor more of the nanoconductors within the connection gap. Thenanoconductors can be suspended in the solvent, or can lie at the bottomof the connection gap on the surface of the chip. Studies have shownthat nanotubes can align both in the suspension and/or on the surface ofthe gap. The electrical conductance of the mixture is less than theelectrical conductance of the nanoconductors within the solution.

The nanoconductors within the connection gap thus experience anincreased alignment in accordance with an increase in the electric fieldapplied across the connection gap. Thus, nanoconnections of theneuron-like node that are utilized most frequently by the neuron-likenode become stronger with each use thereof. The nanoconnections that areutilized least frequently become increasingly weak and eventuallydissolve back into the solution. The nanoconnections may or may notcomprise a resistance, which can be raised or lowered by a selectiveactivation of a nanoconnection. They can be configured as nanoconductorssuch as, for example, a nanotube or nanowire. An example of a nanotube,which may be implemented in accordance with the invention describedherein, is a carbon nanotube, nanowire and/or other nanoparticle.Additionally, such nanoconnections may be configured as a negativeconnection associated with the neuron-like node.

A variable resistor apparatus is also disclosed which includes aplurality of nanoparticles disposed between two terminals, wherein theplurality of nanoparticles provides an electrical resistance. Anelectric field applied to the plurality of nanoparticles across the twoterminals results in an alignment of the nanoparticles over time and adecrease in the electrical resistance thereby providing a variableresistor apparatus. The electric or electrical field can be appliedacross the two terminals perpendicular to the plurality ofnanoconnections. The nanoparticles can comprise nanoconductors, whichcan be formed as, for example, nanotubes and/or nanowires. Thenanoparticles are generally disposed in a solution within a connectiongap formed between the two terminals. The solution can comprise asolvent and/or a suspension of nanoparticles forming a mixture thereof.The solution can also be provided as a liquid, a gel, and or a gas. Thesolution may also comprise a dielectric.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a graph illustrating a typical activation functionthat can be implemented in accordance with one embodiment;

FIG. 2 illustrates a schematic diagram illustrating a diodeconfiguration as a neuron, in accordance with a preferred embodiment;

FIG. 3 illustrates a block diagram illustrating a network of nanowiresbetween two electrodes, in accordance with a preferred embodiment;

FIG. 3 illustrates a block diagram illustrating a network ofnanoconnections formed between two electrodes, in accordance with apreferred embodiment;

FIG. 4 illustrates a block diagram illustrating a plurality ofconnections between inputs and outputs of a physical neural network, inaccordance with a preferred embodiment;

FIG. 5 illustrates a schematic diagram of a physical neural network thatcan be created without disturbances, in accordance with a preferredembodiment;

FIG. 6 illustrates a schematic diagram illustrating an example of aphysical neural network that can be implemented in accordance with analternative embodiment;

FIG. 7 illustrates a schematic diagram illustrating an example of aphysical neural network that can be implemented in accordance with analternative embodiment;

FIG. 8 illustrates a schematic diagram of a chip layout for a connectionnetwork that may be implemented in accordance with an alternativeembodiment;

FIG. 9 illustrates a flow chart of operations illustrating operationalsteps that may be followed to construct a connection network, inaccordance with a preferred embodiment;

FIG. 10 illustrates a flow chart of operations illustrating operationalsteps that may be utilized to strengthen nanoconductors within aconnection gap, in accordance with a preferred embodiment;

FIG. 11 illustrates a schematic diagram of a circuit illustratingtemporal summation within a neuron, in accordance with a preferredembodiment; and

FIG. 12 illustrates a block diagram illustrating a pattern recognitionsystem, which may be implemented with a physical neural network device,in accordance with a preferred embodiment.

DETAILED DESCRIPTION

The particular values and configurations discussed in these non-limitingexamples can be varied and are cited merely to illustrate one or moreembodiments.

The physical neural network described and disclosed herein is differentfrom prior art forms of neural networks in that the disclosed physicalneural network does not require a computer simulation for training, noris its architecture based on any current neural network hardware device.The design of the physical neural network described herein with respectto particular embodiments is actually quite “organic”. Such a physicalneural network is generally fast and adaptable, no matter how large sucha physical neural network becomes. The physical neural network describedherein can be referred to generically as a Knowm. The terms “physicalneural network” and “Knowm” can be utilized interchangeably to refer tothe same device, network, or structure.

Network orders of magnitude larger than current VSLI neural networks canbe built and trained with a standard computer. One consideration for aKnowm is that it must be large enough for its inherent parallelism toshine through. Because the connection strengths of such a physicalneural network are dependant on the physical movement of nanoconnectionsthereof, the rate at which a small network can learn is generally verysmall and a comparable network simulation on a standard computer can bevery fast. On the other hand, as the size of the network increases, thetime to train the device does not change. Thus, even if the networktakes a full second to change a connection value a small amount, if itdoes the same to a billion connections simultaneously, then its parallelnature begins to express itself.

A physical neural network (i.e., a Knowm) must have two components tofunction properly. First, the physical neural network must have one ormore neuron-like nodes that sum a signal and output a signal based onthe amount of input signal received. Such a neuron-like node isgenerally non-linear in its output. In other words, there should be acertain threshold for input signals, below which nothing is output andabove which a constant or nearly constant output is generated or allowedto pass. This is a very basic requirement of standard software-basedneural networks, and can be accomplished by an activation function. Thesecond requirement of a physical neural network is the inclusion of aconnection network composed of a plurality of interconnected connections(i.e., nanoconnections). Such a connection network is described ingreater detail herein.

FIG. 1 illustrates a graph 100 illustrating a typical activationfunction that can be implemented in accordance with one embodiment. Notethat the activation function need not be non-linear, althoughnon-linearity is generally desired for learning complicated input-outputrelationships. The activation function depicted in FIG. 1 comprises alinear function, and is shown as such for general edification andillustrative purposes only. As explained previously, an activationfunction may also be non-linear.

As illustrated in FIG. 1, graph 100 includes a horizontal axis 104representing a sum of inputs, and a vertical axis 102 representingoutput values. A graphical line 106 indicates threshold values along arange of inputs from approximately −10 to +10 and a range of outputvalues from approximately 0 to 1. As more neural networks (i.e., activeinputs) are established, the overall output as indicated at line 105climbs until the saturation level indicated by line 106 is attained. Ifa connection is not utilized, then the level of output (i.e., connectionstrength) begins to fade until it is revived. This phenomenon isanalogous to short term memory loss of a human brain. Note that graph100 is presented for generally illustrative and edification purposesonly and is not considered a limiting feature of the embodiments.

In a Knowm network, the neuron-like node can be configured as a standarddiode-based circuit, the diode being the most basic semiconductorelectrical component, and the signal it sums may be a voltage. Anexample of such an arrangement of circuitry is illustrated in FIG. 2,which generally illustrates a schematic diagram illustrating adiode-based configuration as a neuron 200, in accordance with apreferred embodiment. Those skilled in the art can appreciate that theuse of such a diode-based configuration is not considered a limitationof the embodiments, but merely represents one potential arrangement inwhich the embodiments may be implemented.

Although a diode may not necessarily be utilized, its current versusvoltage characteristics are non-linear when used with associatedresistors and similar to the relationship depicted in FIG. 1. The use ofa diode as a neuron is thus not a limiting feature, but is onlyreferenced herein with respect to a preferred embodiment. The use of adiode and associated resistors with respect to a preferred embodimentsimply represents one potential “neuron” implementation. Such aconfiguration can be said to comprise an artificial neuron. It isanticipated that other devices and components may be utilized instead ofa diode to construct a physical neural network and a neuron-like node(i.e., artificial neuron), as indicated here.

Thus, neuron 200 comprises a neuron-like node that may include a diode206, which is labeled D₁, and a resistor 204, which is labeled R₂.Resistor 204 is connected to a ground 210 and an input 205 of diode 206.Additionally, a resistor 202, which is represented as a block andlabeled R₁ can be connected to input 205 of diode 206. Block 202includes an input 212, which comprises an input to neuron 200. Aresistor 208, which is labeled R₃, is also connected to an output 214 ofdiode 206. Additionally, resistor 208 is coupled to ground 210. Diode206 in a physical neural network is analogous to a neuron of a humanbrain, while an associated connection formed thereof, as explained ingreater detail herein, is analogous to a synapse of a human brain.

As depicted in FIG. 2, the output 214 is determined by the connectionstrength of R₁ (i.e., resistor 202). If the strength of R₁'s connectionincreases (i.e., the resistance decreases), then the output voltage atoutput 214 also increases. Because diode 206 conducts essentially nocurrent until its threshold voltage (e.g., approximately 0.6V forsilicon) is attained, the output voltage will remain at zero until R₁conducts enough current to raise the pre-diode voltage to approximately0.6V. After 0.6V has been achieved, the output voltage at output 214will increase linearly. Simply adding extra diodes in series orutilizing different diode types may increase the threshold voltage.

An amplifier may also be added to the output 214 of diode 206 so thatthe output voltage immediately saturates at the diode threshold voltage,thus resembling a step function, until a threshold value and a constantvalue above the threshold is attained. R₃ (i.e., resistor 208) functionsgenerally as a bias for diode 206 (i.e., D₁) and should generally beabout 10 times larger than resistor 204 (i.e., R₂). In the circuitconfiguration illustrated in FIG. 2, R₁ can actually be configured as anetwork of connections composed of many inter-connected conductingnanowires (i.e., see FIG. 3). As explained previously, such connectionsare analogous to the synapses of a human brain.

FIG. 3 illustrates a block diagram illustrating a network ofnanoconnections 304 formed between two electrodes, in accordance with apreferred embodiment. Nanoconnections 304 (e.g., nanoconductors)depicted in FIG. 3 are generally located between input 302 and output306. The network of nanoconnections depicted in FIG. 3 can beimplemented as a network of nanoconductors. Examples of nanoconductorsinclude devices such as, for example, nanowires, nanotubes, andnanoparticles.

Nanoconnections 304, which are analogous to the synapses of a humanbrain, are preferably composed of electrical conducting material (i.e.,nanoconductors). It should be appreciated by those skilled in the artthat such nanoconductors can be provided in a variety of shapes andsizes without departing from the teachings herein. For example, carbonparticles (e.g., granules or bearings) may be used for developingnanoconnections. The nanoconductors utilized to form a connectionnetwork may be formed as a plurality of nanoparticles.

For example, carbon particles (e.g., granules or bearings) may be usedfor developing nanoconnections. The nanoconductors utilized to form aconnection network may be formed as a plurality of nanoparticles. Forexample, each nanoconnection within a connection network may be formedfrom as a chain of carbon nanoparticles. In “Self-assembled chains ofgraphitized carbon nanoparticles” by Bezryadin et al., Applied PhysicsLetters, Vol. 74, No. 18, pp. 2699-2701, May 3, 1999, for example, atechnique is reported, which permits the self-assembly of conductingnanoparticles into long continuous chains. Thus, nanoconductors whichare utilized to form a physical neural network (i.e., Knowm) could beformed from such nanoparticles. It can be appreciated that the Bezryadinet al is referred to herein for general edification and illustrativepurposes only and is not considered to limit the embodiments.

It can be appreciated that a connection network as disclosed herein maybe composed from a variety of different types of nanoconductors. Forexample, a connection network may be formed from a plurality ofnanoconductors, including nanowires, nanotubes and/or nanoparticles.Note that such nanowires, nanotubes and/or nanoparticles, along withother types of nanoconductors can be formed from materials such ascarbon or silicon. For example, carbon nanotubes may comprise a type ofnanotube that can be utilized in accordance with one or moreembodiments.

As illustrated in FIG. 3, nanoconnections 304 comprise a plurality ofinterconnected nanoconnections, which from this point forward, can bereferred to generally as a “connection network.” An individualnanoconnection may constitute a nanoconductor such as, for example, ananowire, a nanotube, nanoparticles(s), or any other nanoconductingstructures. Nanoconnections 304 may comprise a plurality ofinterconnected nanotubes and/or a plurality of interconnected nanowires.Similarly, nanoconnections 304 may be formed from a plurality ofinterconnected nanoparticles. A connection network is thus not oneconnection between two electrodes, but a plurality of connectionsbetween inputs and outputs. Nanotubes, nanowires, nanoparticles and/orother nanoconducting structures may be utilized, of course, to constructnanoconnections 304 between input 302 and input 306. Although a singleinput 302 and a single input 306 is depicted in FIG. 3, it can beappreciated that a plurality of inputs and a plurality of outputs may beimplemented in accordance with the embodiments, rather than simply asingle input 302 or a single output 306.

FIG. 4 illustrates a block diagram illustrating a plurality ofnanoconnections 414 between inputs 404, 406, 408, 410, 412 and outputs416 and 418 of a physical neural network, in accordance with a preferredembodiment. Inputs 404, 406, 408, 410, and 412 can provide input signalsto connections 414. Output signals can then be generated fromconnections 414 via outputs 416 and 418. A connection network cantherefore be configured from the plurality of connections 414. Such aconnection network is generally associated with one or more neuron-likenodes.

The connection network also comprises a plurality of interconnectednanoconnections, wherein each nanoconnection thereof is strengthened orweakened according to an application of an electric field. A connectionnetwork is not possible if built in one layer because the presence ofone connection can alter the electric field so that other connectionsbetween adjacent electrodes could not be formed. Instead, such aconnection network can be built in layers, so that each connectionthereof can be formed without being influenced by field disturbancesresulting from other connections. This can be seen in FIG. 5.

FIG. 5 illustrates a schematic diagram of a physical neural network 500that can be created without disturbances, in accordance with a preferredembodiment. Physical neural network 500 is composed of a first layer 558and a second layer 560. A plurality of inputs 502, 504, 506, 508, and510 are respectively provided to layers 558 and 560 respectively via aplurality of input lines 512, 514, 516, 518, and 520 and a plurality ofinput lines 522, 524, 526, 528, and 530. Input lines 512, 514, 516, 518,and 520 are further coupled to input lines 532, 534, 536, 538, and 540such that each line 532, 534, 536, 538, and 540 is respectively coupledto nanoconnections 572, 574, 576, 578, and 580. Thus, input line 532 isconnected to nanconnections 572. Input line 534 is connected tonanoconnections 574, and input line 536 is connected to nanoconnections576. Similarly, input line 538 is connected to nanconnections 578, andinput line 540 is connected to nanoconnections 580.

Nanconnections 572, 574, 576, 578, and 580 may comprise nanoconductorssuch as, for example, nanotubes and/or nanowires. Nanoconnections 572,574, 576, 578, and 580 thus comprise one or more nanoconductors.Additionally, input lines 522, 524, 526, 528, and 530 are respectivelycoupled to a plurality of input lines 542, 544, 546, 548 and 550, whichare in turn each respectively coupled to nanoconnections 582, 584, 586,588, and 590. Thus, for example, input line 542 is connected tonanoconnections 582, while input line 544 is connected tonanoconnections 584. Similarly, input line 546 is connected tonanoconnections 586 and input line 548 is connected to nanoconnections588. Additionally, input line 550 is connected to nanconnections 590.Box 556 and 554 generally represent simply the output and are thusillustrated connected to outputs 562 and 568. In other words, outputs556 and 554 respectively comprise outputs 562 and 568. Theaforementioned input lines and associated components thereof actuallycomprise physical electronic components, including conducting input andoutput lines and physical nanoconnections, such as nanotubes and/ornanowires.

Thus, the number of layers 558 and 560 equals the number of desiredoutputs 562 and 568 from physical neural network 500. In the previoustwo figures, every input was potentially connected to every output, butmany other configurations are possible. The connection network can bemade of any electrically conducting material, although the physics of itrequires that they be very small so that they will align with apractical voltage. Carbon nanotubes or any conductive nanowire can beimplemented in accordance with the physical neural network describedherein. Such components can form connections between electrodes by thepresence of an electric field. For example, the orientation andpurification of carbon nanotubes has been demonstrated using acelectrophoresis in isopropyl alcohol, as indicated in “Orientation andpurification of carbon nanotubes using ac electrophoresis” by Yamamotoet al., J. Phys. D: Applied Physics, 31 (1998), 34-36. Additionally, anelectric-field assisted assembly technique used to position individualnanowires suspended in an electric medium between two electrodes definedlithographically on an SiO₂ substrate is indicated in “Electric-fieldassisted assembly and alignment of metallic nanowires,” by Smith et al.,Applied Physics Letters, Vol. 77, Num. 9, Aug. 28, 2000. Such referencesare referred to herein for edification and illustrative purposes only.

The only general requirements for the conducting material utilized toconfigure the nanoconductors are that such conducting material shouldpreferably conduct electricity, and a dipole should preferably beinduced in the material when in the presence of an electric field.Alternatively, the nanoconductors utilized in association with thephysical neural network described herein can be configured to include apermanent dipole that is produced by a chemical means, rather than adipole that is induced by an electric field.

Therefore, it should be appreciated by those skilled in the art that aconnection network could also be comprised of other conductive particlesthat may be developed or found useful in the nanotechnology arts. Forexample, carbon particles (or “dust”) may also be used as nanoconductorsin place of nanowires or nanotubes. Such particles may include bearingsor granule-like particles.

A connection network can be constructed as follows: A voltage is appliedacross a gap that is filled with a mixture of nanowires and a “solvent”.This mixture could be made of many things. The only requirements arethat the conducting wires must be suspended in the solvent, eitherdissolved or in some sort of suspension, free to move around; theelectrical conductance of the substance must be less than the electricalconductance of the suspended conducting wire; and the viscosity of thesubstance should not be too much so that the conducting wire cannot movewhen an electric field is applied.

The goal for such a connection network is to develop a network ofconnections of just the right values so as to satisfy the particularsignal-processing requirement—exactly what a neural network does. Such aconnection network can be constructed by applying a voltage across aspace occupied by the mixture mentioned. To create the connectionnetwork, the input terminals are selectively raised to a positivevoltage while the output terminals are selectively grounded. Thus,connections can gradually form between the inputs and outputs. Theimportant requirement that makes the physical neural network functionalas a neural network is that the longer this electric field is appliedacross a connection gap, or the greater the frequency or amplitude, themore nanotubes and/or nanowires and/or particles align and the strongerthe connection thereof becomes. Thus, the connections that are utilizedmost frequently by the physical neural network become the strongest.

The connections can either be initially formed and have randomresistances or no connections may be formed at all. By initially formingrandom connections, it might be possible to teach the desiredrelationships faster, because the base connections do not have to bebuilt up from scratch. Depending on the rate of connection decay, havinginitial random connections could prove faster, although not necessarily.The connection network can adapt itself to the requirements of a givensituation regardless of the initial state of the connections. Eitherinitial condition will work, as connections that are not used will“dissolve” back into solution. The resistance of the connection can bemaintained or lowered by selective activations of the connection. Inother words, if the connection is not used, it will fade away, analogousto the connections between neurons in a human brain. The temperature ofthe solution can also be maintained at a particular value so that therate that connections fade away can be controlled. Additionally anelectric field can be applied perpendicular to the connections to weakenthem, or even erase them out altogether (i.e., as in clear, zero, orreformatting of a “disk”).

The nanoconnections may or may not be arranged in an orderly arraypattern. The nanoconnections (e.g., nanotubes, nanowires, etc) of aphysical neural network do not have to order themselves into neatlyformed arrays. They simply float in the solution, or lie at the bottomof the gap, and more or less line up in the presence an electric field.Precise patterns are thus not necessary. In fact, neat and precisepatterns may not be desired. Rather, due to the non-linear nature ofneural networks, precise patterns could be a drawback rather than anadvantage. In fact, it may be desirable that the connections themselvesfunction as poor conductors, so that variable connections are formedthereof, overcoming simply an “on” and “off” structure, which iscommonly associated with binary and serial networks and structuresthereof.

FIG. 6 illustrates a schematic diagram illustrating an example of aphysical neural network 600 that can be implemented in accordance withan alternative embodiment. Note that in FIGS. 5 and 6, like parts areindicated by like reference numerals. Thus, physical neural network 600can be configured, based on physical neural network 500 illustrated inFIG. 5. In FIG. 6, inputs 1, 2, 3, 4, and 5 are indicated, which arerespectively analogous to inputs 502, 504, 506, 508, and 510 illustratedin FIG. 5. Outputs 562 and 568 are provided to a plurality of electricalcomponents to create a first output 626 (i.e., Output 1) and a secondoutput 628 (i.e., Output 2). Output 562 is tied to a resistor 606, whichis labeled R2 and a diode 616 at node A. Output 568 is tied to aresistor 610, which is also labeled R2 and a diode 614 at node C.Resistors 606 and 610 are each tied to a ground 602.

Diode 616 is further coupled to a resistor 608, which is labeled R3, andfirst output 626. Additionally, resistor 608 is coupled to ground 602and an input to an amplifier 618. An output from amplifier 618, asindicated at node B and dashed lines thereof, can be tied back to nodeA. A desired output 622 from amplifier 618 is coupled to amplifier 618at node H. Diode 614 is coupled to a resistor 612 at node F. Note thatresistor 612 is labeled R3. Node F is in turn coupled to an input ofamplifier 620 and to second output 628 (i.e., Output 2). Diode 614 isalso connected to second output 628 and an input to amplifier 620 atsecond output 628. Note that second output 628 is connected to the inputto amplifier 620 at node F. An output from amplifier 620 is furthercoupled to node D, which in turn is connected to node C. A desiredoutput 624, which is indicated by a dashed line in FIG. 6, is alsocoupled to an input of amplifier 620 at node E.

In FIG. 6, the training of physical neural network 600 can beaccomplished utilizing, for example, op-amp devices (e.g., amplifiers618 and 620). By comparing an output (e.g., first output 626) ofphysical neural network 600 with a desired output (e.g., desired output622), the amplifier (e.g., amplifier 618) can provide feedback andselectively strengthen connections thereof. For instance, suppose it isdesired to output a voltage of +V at first output 626 (i.e., Output 1)when inputs 1 and 4 are high. When inputs 1 and 4 are taken high, alsoassume that first output 626 is zero. Amplifier 618 can then compare thedesired output (+V) with the actual output (0) and output −V. In thiscase, −V is equivalent to ground.

The op-amp outputs and grounds the pre-diode junction (i.e., see node A)and causes a greater electric field across inputs 1 and 4 and the layer1 output. This increased electric field (larger voltage drop) can causethe nanoconductors in the solution between the electrode junctions toalign themselves, aggregate, and form a stronger connection between the1 and 4 electrodes. Feedback can continue to be applied until output ofphysical neural network 600 matches the desired output. The sameprocedure can be applied to every output.

In accordance with the aforementioned example, assume that Output 1 washigher than the desired output (i.e., desired output 622). If this werethe case, the op-amp output can be +V and the connection between inputs1 and 4 and layer one output can be raised to +V. Columbic repulsionsbetween the nanoconductors can force the connection apart, therebyweakening the connection. The feedback will then continue until thedesired output is obtained. This is just one training mechanism. One cansee that the training mechanism does not require any computations,because it is a simple feedback mechanism.

Such a training mechanism, however, may be implemented in many differentforms. Basically, the connections in a connection network must be ableto change in accordance with the feedback provided. In other words, thevery general notion of connections being strengthened or connectionsbeing weakened in a physical system is the essence of a physical neuralnetwork (i.e., Knowm). Thus, it can be appreciated that the training ofsuch a physical neural network may not require a “CPU” to calculateconnection values thereof. The Knowm can adapt itself. Complicatedneural network solutions could be implemented very rapidly “on the fly”,much like a human brain adapts as it performs.

The physical neural network disclosed herein thus has a number of broadapplications. The core concept of a Knowm, however, is basic. The verybasic idea that the connection values between electrode junctions bynanoconductors can be used in a neural network devise is all thatrequired to develop an enormous number of possible configurations andapplications thereof.

Another important feature of a physical neural network is the ability toform negative connections. This is an important feature that makespossible inhibitory effects useful in data processing. The basic idea isthat the presence of one input can inhibit the effect of another input.In artificial neural networks as they currently exist, this isaccomplished by multiplying the input by a negative connection value.Unfortunately, with a Knowm-based device, the connection may only takeon zero or positive values under such a scenario.

In other words, either there can be a connection or no connection. Aconnection can simulate a negative connection by dedicating a particularconnection to be negative, but one connection cannot begin positive andthrough a learning process change to a negative connection. In general,if starts positive, it can only go to zero. In essence, it is the ideaof possessing a negative connection initially that results in thesimulation, because this does not occur in a human brain. Only one typeof signal travels through axon/dendrites in a human brain. That signalis transferred into the flow of a neurotransmitter whose effect on thepostsynaptic neuron can be either excitatory or inhibitory, depending onthe neuron.

One method for solving this problem is to utilize two sets ofconnections for the same output, having one set represent the positiveconnections and the other set represent the negative connections. Theoutput of these two layers can be compared, and the layer with thegreater output will output either a high signal or a low signal,depending on the type of connection set (inhibitory or excitatory). Thiscan be seen in FIG. 7.

FIG. 7 illustrates a schematic diagram illustrating an example of aphysical neural network 700 that can be implemented in accordance withan alternative embodiment. Physical neural network 700 thus comprises aplurality of inputs 702 (not necessarily binary) which are respectivelyfed to layers 704, 706, 708, and 710. Each layer is analogous to thelayers depicted earlier, such as for example layers 558 and 560 of FIG.5. An output 713 of layer 704 can be connected to a resistor 712, atransistor 720 and a first input 727 of amplifier 726. Transistor 720 isgenerally coupled between ground 701 and first input 727 of amplifier726. Resistor 712 is connected to a ground 701. Note that ground 701 isanalogous to ground 602 illustrated in FIG. 6 and ground 210 depicted inFIG. 2. A second input 729 of amplifier 726 can be connected to athreshold voltage 756. The output of amplifier 726 can in turn be fed toan inverting amplifier 736.

The output of inverting amplifier 736 can then be input to a NOR device740. Similarly, an output 716 of layer 706 may be connected to resistor714, transistor 733 and a first input 733 of an amplifier 728. Athreshold voltage 760 is connected to a second input 737 of amplifier728. Resistor 714 is generally coupled between ground 701 and firstinput 733 of amplifier 728. Note that first input 733 of amplifier 728is also generally connected to an output 715 of layer 706. The output ofamplifier 728 can in turn be provided to NOR device 740. The output fromNOR device 740 is generally connected to a first input 745 of anamplifier 744. An actual output 750 can be taken from first input 745 toamplifier 744. A desired output 748 can be taken from a second input 747to amplifier 744. The output from amplifier 744 is generally provided atnode A, which in turn is connected to the input to transistor 720 andthe input to transistor 724. Note that transistor 724 is generallycoupled between ground 701 and first input 733 of amplifier 728. Thesecond input 731 of amplifier 728 can produce a threshold voltage 760.

Layer 708 provides an output 717 that can be connected to resistor 716,transistor 725 and a first input 737 to an amplifier 732. Resistor 716is generally coupled between ground 701 and the output 717 of layer 708.The first input 737 of amplifier 732 is also electrically connected tothe output 717 of layer 708. A second input 735 to amplifier 732 may betied to a threshold voltage 758. The output from amplifier 732 can inturn be fed to an inverting amplifier 738. The output from invertingamplifier 738 may in turn be provided to a NOR device 742. Similarly, anoutput 718 from layer 710 can be connected to a resistor 719, atransistor 728 and a first input 739 of an amplifier 734. Note thatresistor 719 is generally coupled between node 701 and the output 719 oflayer 710. A second input 741 of amplifier 734 may be coupled to athreshold voltage 762. The output from of NOR device 742 is generallyconnected to a first input 749 of an amplifier 746. A desired output 752can be taken from a second input 751 of amplifier 746. An actual output754 can be taken from first input 749 of amplifier 746. The output ofamplifier 746 may be provided at node B, which in turn can be tied backto the respective inputs to transistors 725 and 728. Note thattransistor 725 is generally coupled between ground 701 and the firstinput 737 of amplifier 732. Similarly, transistor 728 is generallyconnected between ground 701 and the first input 739 of amplifier 734.

Note that transistors 720, 724, 725 and/or 728 each can essentiallyfunction as a switch to ground. A transistor such as, for example,transistor 720, 724, 725 and/or 728 may comprise a field-effecttransistor (FET) or another type of transistor, such as, for example, asingle-electron transistor (SET). Single-electron transistor (SET)circuits are essential for hybrid circuits combining quantum SET deviceswith conventional electronic devices. Thus, SET devices and circuits maybe adapted for use with the physical neural network of the embodiments.This is particularly important because as circuit design rules begin tomove into regions of the sub-100 nanometer scale, where circuit pathsare only 0.001 of the thickness of a human hair, prior art devicetechnologies will begin to fail, and current leakage in traditionaltransistors will become a problem. SET offers a solution at the quantumlevel, through the precise control of a small number of individualelectrons. Transistors such as transistors 720, 724, 725 and/or 728 canalso be implemented as carbon nanotube transistors.

A truth table for the output of circuit 700 is illustrated at block 780in FIG. 7. As indicated at block 780, when an excitatory output is highand the inhibitory output is also high, the final output is low. Whenthe excitatory output is high and the inhibitory output is low, thefinal output is high. Similarly, when the excitatory output is low andthe inhibitory output is high, the final output is low. When theexcitatory output is low and the inhibitory output is also low, thefinal output is low. Note that layers 704 and 708 may thus compriseexcitatory connections, while layers 706 and 710 may comprise inhibitoryconnections.

For every desired output, two sets of connections are used. The outputof a two-diode neuron can be fed into an op-amp (e.g., a comparator). Ifthe output that the op-amp receives is low when it should be high, theop-amp outputs a low signal. This low signal can cause the transistors(e.g., transistors 720, 725) to saturate and ground out the pre-diodejunction for the excitatory diode. Such a scenario can cause, asindicated previously, an increase in the voltage drop across thoseconnections that need to increase their strength. Note that only thoseconnections going to the excitatory diode are strengthened. Likewise, ifthe desired output were low when the actual output was high, the op-ampcan output a high signal. This can cause the inhibitory transistor(e.g., an NPN transistor) to saturate and ground out the neuron junctionof the inhibitory connections. Those connections going to the inhibitorydiode can thereafter strengthen.

At all times during the learning process, a weak alternating electricfield can be applied perpendicular to the connections. This can causethe connections to weaken by rotating the nanotube perpendicular to theconnection direction. This perpendicular field is important because itcan allow for a much higher degree of adaptation. To understand this,one must realize that the connections cannot (practically) keep gettingstronger and stronger. By weakening those connections not contributingmuch to the desired output, we decrease the necessary strength of theneeded connections and allow for more flexibility in continuoustraining. This perpendicular alternating voltage can be realized by theaddition of two electrodes on the outer extremity of the connection set,such as plates sandwiching the connections (i.e., above and below).Other mechanisms, such as increasing the temperature of the nanotubesuspension could also be used for such a purpose, although this methodis perhaps a little less controllable or practical.

The circuit depicted in FIG. 7 can be separated into two separatecircuits. The first part of the circuit can be composed of nanotubeconnections, while the second part of the circuit comprises the“neurons” and the learning mechanism (i.e., op-amps/comparator). Thelearning mechanism on first glance appears similar to a relativelystandard circuit that could be implemented on silicon with currenttechnology. Such a silicon implementation can thus comprise the “neuron”chip. The second part of the circuit (i.e., the connections) is thus anew type of chip, although it could be constructed with currenttechnology. The connection chip can be composed of an orderly array ofelectrodes spaced anywhere from, for example, 100 nm to 1 μm or perhapseven further. In a biological system, one talks of synapses connectingneurons. It is in the synapses where the information is processed,(i.e., the “connection weights”). Similarly, such a chip can contain allof the synapses for the physical neural network. A possible arrangementthereof can be seen in FIG. 8.

FIG. 8 illustrates a schematic diagram of a chip layout 800 for aconnection network that may be implemented in accordance with analternative embodiment. FIG. 8 thus illustrates a possible chip layoutfor a connection chip (i.e., connection network 800) that can beimplemented in accordance with one or more embodiments. Chip layout 800includes an input array composed of plurality of inputs 801, 802, 803,804, and 805, which are provided to a plurality of layers 806, 807, 808,809, 810, 811, 812, 813, 814, and 815. A plurality of outputs 802 can bederived from layers 806, 807, 808, 809, 810, 811, 812, 813, 814, and815. Inputs 801 can be coupled to layers 806 and 807, while inputs 802can be connected to layers 808 and 809. Similarly, inputs 803 can beconnected to layers 810 and 811. Also, inputs 804 can be connected tolayers 812 and 813. Inputs 805 are generally connected to layers 814 and815.

Similarly, such an input array can includes a plurality of inputs 831,832, 833, 834 and 835 which are respectively input to a plurality oflayers 816, 817, 818, 819, 820, 821, 822, 823, 824 and 825. Thus, inputs831 can be connected to layers 816 and 817, while inputs 832 aregenerally coupled to layers 818 and 819. Additionally, inputs 833 can beconnected to layers 820 and 821. Inputs 834 can be connected to layers822 and 823. Finally, inputs 835 are connected to layers 824 and 825.Arrows 828 and 830 represent a continuation of the aforementionedconnection network pattern. Those skilled in the art can appreciate, ofcourse, that chip layout 800 is not intended to represent an exhaustivechip layout or to limit the scope of the invention. Many modificationsand variations to chip layout 800 are possible in light of the teachingsherein without departing from the scope of the embodiments. It iscontemplated that the use of a chip layout, such as chip layout 800, caninvolve a variety of components having different characteristics.

Preliminary calculations based on a maximum etching capability of 200 nmresolution indicated that over 4 million synapses could fit on an areaof approximately 1 cm². The smallest width that an electrode can possessis generally based on current lithography. Such a width may of coursechange as the lithographic arts advance. This value is actually about 70nm for state-of-the-art techniques currently. These calculations are ofcourse extremely conservative, and are not considered a limiting featureof the embodiments. Such calculations are based on an electrode with,separation, and gap of approximately 200 nm. For such a calculation, forexample, 166 connection networks comprising 250 inputs and 100 outputscan fit within a one square centimeter area.

If such chips are stacked vertically, an untold number of synapses couldbe attained. This is two to three orders of magnitude greater than someof the most capable neural network chips out there today, chips thatrely on standard methods to calculate synapse weights. Of course, thegeometry of the chip could take on many different forms, and it is quitepossible (based on a conservative lithography and chip layout) that manymore synapses could fit in the same space. The training of a chip thissize would take a fraction of the time of a comparably sized traditionalchip using digital technology.

The training of such a chip is primarily based on two assumptions.First, the inherent parallelism of a physical neural network (i.e., aKnowm) can permit all training sessions to occur simultaneously, nomatter now large the associated connection network. Second, recentresearch has indicated that near perfect aligning of nanotubes can beaccomplished in approximately 15 minutes. If one considers that theinput data, arranged as a vector of binary “high's” and “low's” ispresented to the Knowm simultaneously, and that all training vectors arepresented one after the other in rapid succession (e.g., perhaps 100 MHzor more), then each connection would “see” a different frequency indirect proportion to the amount of time that its connection is requiredfor accurate data processing (i.e., provided by a feedback mechanism).Thus, if it only takes approximately 15 minutes to attain an almostperfect state of alignment, then this amount of time would comprise thelongest amount of time required to train, assuming that all of thetraining vectors are presented during that particular time period.

FIG. 9 illustrates a flow chart 900 of operations illustratingoperational steps that may be followed to construct a connectionnetwork, in accordance with a preferred embodiment. Initially, asindicated at block 902, a connection gap is created from a connectionnetwork structures. As indicated earlier, the goal for such a connectionnetwork is generally to develop a network of connections of “just” theright values to satisfy particular information processing requirements,which is precisely what a neural network accomplishes. As illustrated atblock 904, a solution is prepared, which is composed of nanoconductorsand a “solvent.” Note that the term “solvent” as utilized herein has avariable meaning, which includes the traditional meaning of a “solvent,”and also a suspension.

The solvent utilized can comprise a volatile liquid that can be confinedor sealed and not exposed to air. For example, the solvent and thenanoconductors present within the resulting solution may be sandwichedbetween wafers of silicon or other materials. If the fluid has a meltingpoint that is approximately at room temperature, then the viscosity ofthe fluid could be controlled easily. Thus, if it is desired to lock theconnection values into a particular state, the associated physicalneural network (i.e., Knowm) may be cooled slightly until the fluidfreezes. The term “solvent” as utilized herein thus can include fluidssuch as for example, toluene, hexadecane, mineral oil, etc. Note thatthe solution in which the nanoconductors (i.e., nanoconnections) arepresent should generally comprise a dielectric. Thus, when theresistance between the electrodes is measured, the conductivity of thenanoconductors can be essentially measured, not that of the solvent. Thenanoconductors can be suspended in the solution or can alternately lieon the bottom surface of the connection gap. The solvent may also beprovided in the form of a gas.

As illustrated thereafter at block 906, the nanoconductors must besuspended in the solvent, either dissolved or in a suspension of sorts,but generally free to move around, either in the solution or on thebottom surface of the gap. As depicted next at block 908, the electricalconductance of the solution must be less than the electrical conductanceof the suspended nanoconductor(s). Similarly, the electrical resistanceof the solution is greater than the electrical resistance of thenanoconductor.

Next, as illustrated at block 910, the viscosity of the substance shouldnot be too much so that the nanoconductors cannot move when an electricfield (e.g., voltage) is applied. Finally, as depicted at block 912, theresulting solution of the “solvent” and the nanoconductors is thuslocated within the connection gap.

Note that although a logical series of steps is illustrated in FIG. 9,it can be appreciated that the particular flow of steps can bere-arranged. Thus, for example, the creation of the connection gap, asillustrated at block 902, may occur after the preparation of thesolution of the solvent and nanoconductor(s), as indicated at block 904.FIG. 9 thus represents merely possible series of steps, which may befollowed to create a connection network. A variety of other steps may befollowed as long as the goal of achieving a connection network isachieved. Similar reasoning also applies to FIG. 10.

FIG. 10 illustrates a flow chart 1000 of operations illustratingoperational steps that may be utilized to strengthen nanoconductorswithin a connection gap, in accordance with a preferred embodiment. Asindicated at block 1002, an electric field can be applied across theconnection gap discussed above with respect to FIG. 9. The connectiongap can be occupied by the solution discussed above. As indicatedthereafter at block 1004, to create the connection network, the inputterminals can be selectively raised to a positive voltage while theoutput terminals are selectively grounded. As illustrated thereafter atblock 1006, connections thus form between the inputs and the outputs.The important requirements that make the resulting physical neuralnetwork functional as a neural network is that the longer this electricfield is applied across the connection gap, or the greater the frequencyor amplitude, the more nanoconductors align and the stronger theconnection becomes. Thus, the connections that get utilized the mostfrequently become the strongest.

As indicated at block 1008, the connections can either be initiallyformed and have random resistances or no connections will be formed atall. By forming initial random connections, it might be possible toteach the desired relationships faster, because the base connections donot have to be built up as much. Depending on the rate of connectiondecay, having initial random connections could prove to be a fastermethod, although not necessarily. A connection network can adapt itselfto whatever is required regardless of the initial state of theconnections. Thus, as indicated at block 1010, as the electric field isapplied across the connection gap, the more the nonconductor(s) willalign and the stronger the connection becomes. Connections (i.e.,synapses) that are not used are dissolved back into the solution, asillustrated at block 1012. As illustrated at block 1014, the resistanceof the connection can be maintained or lowered by selective activationsof the connections. In other words, “if you do not use the connection,it will fade away,” much like the connections between neurons in a humanbrain.

The neurons in a human brain, although seemingly simple when viewedindividually, interact in a complicated network that computes with bothspace and time. The most basic picture of a neuron, which is usuallyimplemented in technology, is a summing device that adds up a signal.Actually, this statement can be made even more general by stating that aneuron adds up a signal in discrete units of time. In other words, everygroup of signals incident upon the neuron can be viewed as occurring inone moment in time. Summation thus occurs in a spatial manner. The onlydifference between one signal and another signal depends on where suchsignals originate. Unfortunately, this type of data processing excludesa large range of dynamic, varying situations that cannot necessarily bebroken up into discrete units of time.

The example of speech recognition is a case in point. Speech occurs inthe time domain. A word is understood as the temporal pronunciation ofvarious syllables. A sentence is composed of the temporal separation ofvarying words. Thoughts are composed of the temporal separation ofvarying sentences. Thus, for an individual to understand a spokenlanguage at all, a syllable, word, sentence or thought must exert sometype of influence on another syllable, word, sentence or thought. Themost natural way that one sentence can exert any influence on anothersentence, in the light of neural networks, is by a form of temporalsummation. That is, a neuron “remembers” the signals it received in thepast.

The human brain accomplishes this feat in an almost trivial manner. Whena signal reaches a neuron, the neuron has an influx of ions rush throughits membrane. The influx of ions contributes to an overall increase inthe electrical potential of the neuron. Activation is achieved when thepotential inside the cell reaches a certain threshold. The one caveat isthat it takes time for the cell to pump out the ions, something that itdoes at a more or less constant rate. So, if another signal arrivesbefore the neuron has time to pump out all of the ions, the secondsignal will add with the remnants of the first signal and achieve araised potential greater than that which could have occurred with onlythe second signal. The first signal influences the second signal, whichresults in temporal summation.

Implementing this in a technological manner has proved difficult in thepast. Any simulation would have to include a “memory” for the neuron. Ina digital representation, this requires data to be stored for everyneuron, and this memory would have to be accessed continually. In acomputer simulation, one must discritize the incoming data, sinceoperations (such as summations and learning) occur serially. That is, acomputer can only do one thing at a time. Transformations of a signalfrom the time domain into the spatial domain require that time be brokenup into discrete lengths, something that is not necessarily possiblewith real-time analog signals in which no point exists within atime-varying signal that is uninfluenced by another point.

A physical neural network, however, is generally not digital. A physicalneural network is a massively parallel analog device. The fact thatactual molecules (e.g., nanoconductors) must move around (in time) makestemporal summation a natural occurrence. This temporal summation isbuilt into the nanoconnections. The easiest way to understand this is toview the multiplicity of nanoconnections as one connection with oneinput into a neuron-like node (Op-amp, Comparator, etc.). This can beseen in FIG. 11.

FIG. 11 illustrates a schematic diagram of a circuit 1100 illustratingtemporal summation within a neuron, in accordance with a preferredembodiment. As indicated in FIG. 11, an input 1102 can be provided tonanoconnections 1104, which in turn can provide a signal, which can beinput to an amplifier 1110 (e.g., op amp) at node B. A resistor 1106 canbe connected to node A, which in turn is electrically equivalent to nodeB. Node B can be connected to a negative input of amplifier 1100.Resistor 1108 can also be connected to a ground 1108. Amplifier 1110provides output 1114. Note that although nanoconnections 1104 isreferred to in the plural it can be appreciated that nanoconnections1104 can comprise a single nanoconnection or a plurality ofnanoconnections. For simplicity sake, however, the plural form is usedto refer to nanoconnections 1104.

Input 1102 can be provided by another physical neural network (i.e.,Knowm) to cause increased connection strength of nanoconnections 1104over time. This input would most likely arrive in pulses, but could alsobe continuous. A constant or pulsed electric field perpendicular to theconnections can serve to constantly erode the connections, so that onlysignals of a desired length or amplitude can cause a connection to form.Once the connection is formed, the voltage divider formed bynanoconnection 1104 and resistor 1106 can cause a voltage at node A indirect proportion to the strength of nanoconnections 1104. When thevoltage at node A reaches a desired threshold, the amplifier (i.e., anop-amp and/or comparator), will output a high voltage (i.e., output1114). The key to the temporal summation is that, just like a realneuron, it takes time for the electric field to breakdown thenanoconnections 1104, so that signals arriving close in time willcontribute to the firing of the neuron (i.e., op-amp, comparator, etc.).Temporal summation has thus been achieved. The parameters of thetemporal summation could be adjusted by the amplitude and frequency ofthe input signals and the perpendicular electric field.

FIG. 12 illustrates a block diagram illustrating a pattern recognitionsystem 1200, which may be implemented with a physical neural networkdevice 1222, in accordance with an alternative embodiment. Note thatpattern recognition system 1200 can be implemented as a speechrecognition system. Although pattern recognition system 1200 is depictedherein in the context of speech recognition, a physical neural networkdevice (i.e., a Knowm device) may be implemented with other patternrecognition systems, such as visual and/or imaging recognition systems.FIG. 12 thus does not comprise a limiting feature of the embodiments andis presented for general edification and illustrative purposes only.Those skilled in the art can appreciate that the diagram depicted inFIG. 12 may be modified as new applications and hardware are developed.The development or use of a pattern recognition system such as patternrecognition system 1200 of FIG. 12 by no means limits the scope of thephysical neural network (i.e., Knowm) disclosed herein.

FIG. 12 thus illustrates in block diagram fashion, the system structureof a speech recognition device using a neural network according to analternative embodiment. The pattern recognition system 1200 can beprovided with a CPU 1211 for performing the functions of inputtingvector rows and instructor signals (vector rows) to an output layer forthe learning process of a physical neural network device 1222, andchanging connection weights between respective neuron devices based onthe learning process. Pattern recognition system 1200 can be implementedwithin the context of a data-processing system, such as, for example, apersonal computer or personal digital assistant (PDA), both of which arewell known in the art.

The CPU 1211 can perform various processing and controlling functions,such as pattern recognition, including but not limited to speech and/orvisual recognition based on the output signals from the physical neuralnetwork device 1222. The CPU 1211 is connected to a read-only memory(ROM) 1213, a random-access memory (RAM) 1214, a communication controlunit 1215, a printer 1216, a display unit 1217, a keyboard 1218, an FFT(fast Fourier transform) unit 1221, a physical neural network device1222 and a graphic reading unit 1224 through a bus line 1220 such as adata bus line. The bus line 1220 may comprise, for example, an ISA,EISA, or PCI bus.

The ROM 1213 is a read-only memory storing various programs or data usedby the CPU 1211 for performing processing or controlling the learningprocess, and speech recognition of the physical neural network device1222. The ROM 1213 may store programs for carrying out the learningprocess according to error back-propagation for the physical neuralnetwork device or code rows concerning, for example, 80 kinds ofphonemes for performing speech recognition. The code rows concerning thephonemes can be utilized as second instructor signals and forrecognizing phonemes from output signals of the neuron device network.Also, the ROM 1213 can store programs of a transformation system forrecognizing speech from recognized phonemes and transforming therecognized speech into a writing (i.e., written form) represented bycharacters.

A predetermined program stored in the ROM 1213 can be downloaded andstored in the RAM 1214. RAM 1214 generally functions as a random accessmemory used as a working memory of the CPU 1211. In the RAM 1214, avector row storing area can be provided for temporarily storing a powerobtained at each point in time for each frequency of the speech signalanalyzed by the FFT unit 1221. A value of the power for each frequencyserves as a vector row input to a first input portion of the physicalneural network device 1222. Further, in the case where characters orgraphics are recognized in the physical neural network device, the imagedata read by the graphic reading unit 1224 are stored in the RAM 1214.

The communication control unit 1215 transmits and/or receives variousdata such as recognized speech data to and/or from another communicationcontrol unit through a communication network 1202 such as a telephoneline network, an ISDN line, a LAN, or a personal computer communicationnetwork. Network 1202 may also comprise, for example, atelecommunications network, such as a wireless communications network.Communication hardware methods and systems thereof are well known in theart.

The printer 1216 can be provided with a laser printer, a bubble-typeprinter, a dot matrix printer, or the like, and prints contents of inputdata or the recognized speech. The display unit 1217 includes an imagedisplay portion such as a CRT display or a liquid crystal display, and adisplay control portion. The display unit 1217 can display the contentsof the input data or the recognized speech as well as a direction of anoperation required for speech recognition utilizing a graphical userinterface (GUI).

The keyboard 1218 generally functions as an input unit for varyingoperating parameters or inputting setting conditions of the FFT unit1221, or for inputting sentences. The keyboard 1218 is generallyprovided with a ten-key numeric pad for inputting numerical figures,character keys for inputting characters, and function keys forperforming various functions. A mouse 1219 can be connected to thekeyboard 1218 and serves as a pointing device.

A speech input unit 1223, such as a microphone can be connected to theFFT unit 1221. The FFT unit 1221 transforms analog speech data inputfrom the voice input unit 1223 into digital data and carries outspectral analysis of the digital data by discrete Fouriertransformation. By performing a spectral analysis using the FFT unit1221, the vector row based on the powers of the respective frequenciesare output at predetermined intervals of time. The FFT unit 1221performs an analysis of time-series vector rows, which representcharacteristics of the inputted speech. The vector rows output by theFFT 1221 are stored in the vector row storing area in the RAM 1214.

The graphic reading unit 224, provided with devices such as a CCD(Charged Coupled Device), can be used for reading images such ascharacters or graphics recorded on paper or the like. The image dataread by the image-reading unit 1224 are stored in the RAM 1214. Notethat an example of a pattern recognition apparatus, which may bemodified for use with the physical neural network described herein, isdisclosed in U.S. Pat. No. 6,026,358 to Tomabechi, Feb. 16, 2000,“Neural Network, A Method of Learning of a Neural Network and PhonemeRecognition Apparatus Utilizing a Neural Network.” U.S. Pat. No.6,026,358 is incorporated herein by reference. It can be appreciatedthat the Tomabechi reference does not teach, suggest or anticipate theembodiments, but is discussed herein for general illustrative,background and general edification purposes only.

The implications of a physical neural network are tremendous. Withexisting lithography technology, many electrodes in an array such asdepicted in FIG. 5 can be etched onto a wafer of silicon. Theneuron-diodes, as well as the training circuitry illustrated in FIG. 6,could be built onto the same silicon wafer, although it may be desirableto have the connections on a separate chip due to the liquid solution ofnanoconductors. A solution of suspended nanoconductors could be placedbetween the electrode connections and the chip could be packaged. Theresulting “chip” would look much like a current Integrated Chip (IC) orVLSI (very large scale integrated) chips. One could also place a ratherlarge network parallel with a computer processor as part of a largersystem. Such a network, or group of networks, could add significantcomputational capabilities to standard computers and associatedinterfaces.

For example, such a chip may be constructed utilizing a standardcomputer processor in parallel with a large physical neural network orgroup of physical neural networks. A program can then be written suchthat the standard computer teaches the neural network to read, or createan association between words, which is precisely the same sort of taskin which neural networks can be implemented. Once the physical neuralnetwork is able to read, it can be taught for example to “surf” theInternet and find material of any particular nature. A search engine canthen be developed that does not search the Internet by “keywords”, butinstead by meaning. This idea of an intelligent search engine hasalready been proposed for standard neural networks, but until now hasbeen impractical because the network required was too big for a standardcomputer to simulate. The use of a physical neural network (i.e.,physical neural network) as disclosed herein now makes a trulyintelligent search engine possible.

A physical neural network can be utilized in other applications, suchas, for example, speech recognition and synthesis, visual and imageidentification, management of distributed systems, self-driving cars,filtering, etc. Such applications have to some extent already beenaccomplished with standard neural networks, but are generally limited inexpense, practicality and not very adaptable once implemented. The useof a physical neural network can permit such applications to become morepowerful and adaptable. Indeed, anything that requires a bit more“intelligence” could incorporate a physical neural network. One of theprimary advantages of a physical neural network is that such a deviceand applications thereof can be very inexpensive to manufacture, evenwith present technology. The lithographic techniques required forfabricating the electrodes and channels therebetween has already beenperfected and implemented in industry.

Most problems in which a neural network solution is implemented arecomplex adaptive problems, which change in time. An example is weatherprediction. The usefulness of a physical neural network is that it couldhandle the enormous network needed for such computations and adaptitself in real-time. An example wherein a physical neural network (i.e.,Knowm) can be particularly useful is the Personal Digital Assistant(PDA). PDA's are well known in the art. A physical neural networkapplied to a PDA device can be advantageous because the physical neuralnetwork can ideally function with a large network that could constantlyadapt itself to the individual user without devouring too muchcomputational time from the PDA. A physical neural network could also beimplemented in many industrial applications, such as developing areal-time systems control to the manufacture of various components. Thissystems control can be adaptable and totally tailored to the particularapplication, as necessarily it must.

It will be appreciated that variations of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be desirablycombined into many other different systems or applications. Also thatvarious presently unforeseen or unanticipated alternatives,modifications, variations or improvements therein may be subsequentlymade by those skilled in the art which are also intended to beencompassed by the following claims.

1. A variable resistor apparatus, comprising: a plurality ofnanoparticles disposed between two terminals, wherein said plurality ofnanoparticles provide an electrical resistance; and wherein an electricfield applied to said plurality of nanoparticles across said twoterminals results in an alignment of said nanoparticles over time and adecrease in said electrical resistance thereby providing a variableresistor apparatus.
 2. The apparatus of claim 1 wherein said electricfield is applied across said two terminals perpendicular to saidplurality of nanoconnections.
 3. The apparatus of claim 1 wherein saidnanoparticles among said plurality of nanoparticles comprisenanoconductors.
 4. The apparatus of claim 1 wherein said nanoconductorscomprise nanotubes.
 5. The apparatus of claim 1 wherein saidnanoconductors comprise nanowires.
 6. The apparatus of claim 1 whereinsaid plurality of nanoparticles are disposed in a solution within aconnection gap formed between said two terminals.
 7. The apparatus ofclaim 7 wherein said solution comprises a solvent.
 8. The apparatus ofclaim 7 wherein said solution comprises a suspension of saidnanoparticles forming a mixture thereof.
 9. The apparatus of claim 7wherein said solution comprises a liquid.
 10. The apparatus of claim 7wherein said solution comprises a gel.
 11. The apparatus of claim 7wherein said solution comprises a gas.
 12. The apparatus of claim 7wherein said solution comprises a dielectric.
 13. A variable resistorapparatus, comprising: a plurality of nanoparticles disposed between ina solution within a connection gap formed between two terminals, whereinsaid plurality of nanoparticles provide an electrical resistance; andwherein an electric field applied to said plurality of nanoparticlesacross said two terminals perpendicular to said plurality ofnanoconnections results in an alignment of said nanoparticles over timeand a decrease in said electrical resistance thereby providing avariable resistor apparatus.
 14. The apparatus of claim 13 wherein saidsolution comprises a solvent.
 15. The apparatus of claim 13 wherein saidsolution comprises a suspension of said nanoparticles forming a mixturethereof.
 16. The apparatus of claim 13 wherein said solution comprisesat least one of the following: a liquid, a gel, a gas or a dielectric.17. A variable resistor apparatus, comprising: a plurality ofnanoconductors disposed between in a solution within a connection gapformed between two terminals, wherein said plurality of nanoconductorsprovide an electrical resistance; and wherein an electric field appliedto said plurality of nanoconductors across said two terminalsperpendicular to said plurality of nanoconnections results in analignment of said nanoconductors over time and a decrease in saidelectrical resistance thereby providing a variable resistor apparatus.